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(54) Title: PRINTABLE INTERFACES AND DIGITAL LINKMARKS 
(57) Abstract 

Hie present invention improves die 
human/computer interface by providing printable 
interfaces that enable a user to invoke and control 
computer processes. TTic present invention teaches 
an encoded physical medium suitable for use in 
interfacing a computer user and a computer system 
such that the user can control and/or access a 
phirality of computer implemented processes such 
as computer applications and vi^cb pages. The 
encoded physical medium has a Unkmark and 
an instruction mark. Encoded within the Unkmaik 
is machine readable linking infomiation directing 
to a computer implemented process. Encoded 
within the instruction mark is a machine readable 
operating instruction that, when decoded, may be 
executed by the computer implemented process. 
The present invention also teaches diat the marks 
can present human readable infomiaUon related 
to the nauire of die machine readable information 
stored therein. For example, the Linkmark may 
have graphical illustrations indicating content 
found at die web page, media format of die 
content, user designation, payment and membership 
requirements, communication speed, duration of 
content, etc. The present invention ftirther teaches 
a two-dimensional bar code with a plurality of 
colored glyphs formed such that the average 
color of the two-^Iimcnsional bar code represents 
encoded mformaUon. The colored glyphs are also 
arranged such that digital information is encoded 
withm the two-dimensional bar code. 
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PRINTABLE INTERFACES AND DIGITAL LINKIMARKS 
Background of the Invention 

5 This invention relates generally to computer interfaces. More specifically, 

the present invention discloses a variety of printable interfaces wherein an encoded 
Linkmark™ provides machine readable information and a Multicon™ Linkmark 
provides both machine and human readable information. The human readable 
information indicates to the user the nature of the machine readable information, 
10 and the machine information enables a user to interface with and control a 
computer system. The present invention also teaches multi-colored two- 
dimensional bar codes that provide machine readable information through both the 
digital information in the bar code glyph and the average color of the bar code 
glyph. 

15 People are constantly interacting with computerized systems, from the 

trivial {e,g.^ the computerized toaster or the remote control television) to the 
exceedingly complex {e.g,y telecommunications systems and the Internet). An 
advantage of computerization is that such systenfis provide flexibility and power to 
their users. However, the price that must be paid for this power and flexibility is, 

20 typically, an increase in the difficulty of the human/machine interface. 

A fundamental reason for this problem is that computers operate on 
principles based on the abstract concepts of mathematics and logic, while humans 
tend to think in a more spatial manner. Often people are more comfortable with 
physical, three-dimensional objects than they are with the abstractions of the 

25 computer world. In short, the power and flexibility provided by flie computer and 
related electronic technology are inherently limited by the ability of the human 
user to control these devices. Since people do not think like computers, metaphors 
are adopted to permit people to effectively conraiunicate with computers. In 
general, better metaphors permit more efficient and medium independent 

30 communications between people and computers. The better metaphor will provide 
the user a natural and intuitive interface with the computer without sacrificing the 
computer's potential. 

There are, of course, a number of computer interfaces which allow users, 
with varying degrees of comfort and ease, to interact with computers. For 
35 example, keyboards, computer mice, joysticks, etc. allow users to physically 
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manipulate a three-dimensional object to create an input into a computer system. 
However, these computer interfaces are quite artificial in nature, and tend to 
require a substantial investment in training to be used efficiently. 

Progress has been made in improving the computer interface with the 
graphical user interface (GUI). With a GUI, icons that represent physical objects 
are displayed on a computer screen. For example, a document file may look like a 
page of a document, a directory file might look like a file folder, and an icon of a 
trash can may be used for disposing of documents and files. In other words, GUIs 
use "metaphors" where a graphical icon represents a physical object familiar to 
users. This makes GUIs easier for most people to use. GUIs were pioneered at 
such places as Xerox PARC of Palo Alto, California and Apple Computer, Inc. of 
Cupertino, California. The GUI is also often commonly used with UNDC^^ based 
systems, and is rapidly becoming a standard in the PC/MS-DOS world with the 
Windows™ operating system provided by Microsoft Corporation of Redmond, 
Washington. 

While GUIs are a major advance in computer interfaces, they nonetheless 
present a user with a leaming curve due to their still limited metaphor. In other 
words, an icon can only represent a physical object; it is not itself a physical 
object. It would be ideal if the computer interface was embodied in a physical 
medium which could convey a familiar meaning, one perhaps relevant to the task 
at hand. While progress has been made towards achieving such a goal, many 
roadblocks yet remain. For example, assuming that for a given application one has 
selected a physical medium for use as a computer interface, the information 
necessary to support the computer interface must still be encoded within the 
physical medium. Additionally, techniques must be developed for linking such 
interfaces with the vast wealth of information available from remote sources using 
computer networks like the Intemet. 

Redford et al.'s U.S. Patent Serial No. 5,634,265, entitled "PRINTED 
PUBLICATION REMOTE CONTROL FOR ACCESSING INTERACTIVE 
MEDIA," filed July 1, 1994, describes one rudimentary mechanism for encoding 
information within a physical medium. Redford describes the use of a printed 
publication such as a book being constructed to include a storage media, a data 
button, and remote control circuitry. The button is physically attached to the 
printed publication and when activated by a user, data fi-om the storage media can 
initiate local feedback at the printed publication and the remote control can 
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transmit a control message to a remote computer system which in turn performs 
some desired operation. 

While strides have been made in attempting to improve computer interfaces, 
there is still progress to be made in this field. Ultimately, the interface itself 
should disappear from the conscious thought of users so that they can intuitively 
accomplish their goals without concern to the mechanics of the interface or the 
underlying operation of the computerized system. 
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Summary of fhe Invention 

The present invention improves the human/computer interface by providing 
printable interfaces that enable a user to invoke and control computer processes. A 
first embodiment of the present invention teaches an encoded physical medium 
suitable for use in interfacing a computer user and a computer system such that the 
user can control a plurality of computer implemented processes. The encoded 
physical medium has a Linkmark and an instruction mark. Encoded within the 
Linkmark is machine readable linking information directing to a computer 
implemented process. Encoded within the instruction mark is a machine readable 
operating instruction that, when decoded, may be executed by the computer 
implemented process. 

The computer implemented process may be a web site, a web page, a 
computer application, etc. Hence, when the computer implemented process is a 
web page or web site, the machine readable linking information provides, either 
directly or indirectly by way of reference to a central server database, a uniform 
resource locator (URL) pointing to the web page or web site. In this case, the 
machine readable operating instruction could be a navigation command or other 
suitable instruction for controlling the invoked web site. 

Certain embodiments of the present invention teach that certain marks 
(termed Multicon Linkmarks) also present human readable information related to 
the nature of the machine readable information stored therein. For example, the 
Multicon Linkmark may have graphical illustrations indicating content found at 
the web page, media format of the content, user designation, payment and 
membership requirements, communication speed, duration of content, etc. 

In another embodiment, the format of the machine readable infonnation is a 
two-dimensional bar code comprising a plurality of colored glyphs, the plurality of 
colored glyphs arranged such that digital information is encoded within the two- 
dimensional bar code, the color intensities of the colored glyphs being such that 
ftirther information is encoded as an average color intensity of the combination of 
the plurality of colored glyphs. 

Still another embodiment of the present invention teaches a computer 
system suitable for controlling a computer implemented process. The computer 
system has a digital processor, memory coupled to die digital processor storing 
operating instructions suitable for controlling the operation of the digital 
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processor, an encoded physical medium similar to that described above, and a 
sensor coupled to the digital processor and operable for measuring information 
present within each mark located on the encoded physical medium. Thus when a 
user engages the sensor with one of the encoded marks located on the physical 
medium, the computer system responds by performing the appropriate action such 
as invoking the indicated computer process or executing the instruction within the 
computer process. 

The present invention therefore provides a more intuitive and richer 
metaphor for the interaction between humans and computerized systems. These 
and other advantages of the present invention will become apparent upon reading 
the following detailed descriptions and studying the various figures of the 
drawings. 



--5" 
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Brief Description of the Drawing s 



Figure 1 is a diagrammatic illustration of a computer interface in accordance 
with one embodiment of the present invention. 

Figure 2 is a flow chart illustrating one method for providing an interface 
between a user and a computer system in accordance with one aspect of the 
present invention. 

Figures 3-5 are illustrations of three Linkmarks in accordance with three 
related embodiments of the present invention. 

Figure 6 is a diagrammatic illustration of a hot spot in accordance with one 
embodiment of the present invention. 

Figure 7 illustrates a sensor responsive to a spectral encoding scheme in 
accordance with yet another embodiment of the present invention. 

Figure 8 is a flow chart illustrating one suitable method for measuring 
information encoded in a hot spot. 

Figure 9 is a flow chart illustrating one suitable method for invoking and 
controlling a computer implemented process with Linkmarks and instruction 
marks in accordance with one aspect of the present invention; 

Figure 10 illustrates a Multicon Linkmark including both machine and 
human readable information, the human readable information being related to the 
characteristic traits of the computer process indicated by the machine readable 
information; 

Figure II illustrates a printable interface having the Multicon Linkmark of 
Figure 10 together with four different instruction Linkmarks; 

Figure 12 is a Multicon legend showing a variety of icons each providing a 
graphical illustration of a certain characteristic trait of a corresponding web site; 

Figures 13-15 illustrate three different Multicon Linkmarks demonstrating 
the encoding scheme of the Multicon legend of Figure 12; and 
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Figure 16 illustrates a two-dimensional bar code in accordance with one 
embodiment of the present invention. 
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Detailed Description of the Preferred Embodiments 

With reference to Figure 1, a computer interface 10 in accordance with one 
embodiment of the present invention will now be described. The interface 10 
includes a sensor 12 and an encoded physical medium 30. As will be apparent, the 
computer interface 10 provides an intuitive mechanism for a user to interface with 
and control an electronic device such as a computer system 40, As further 
illustrated in Figure 1, the computer system 40 is coupled bi-directionally to a 
network such as the Internet 42 via communications link 44. Those skilled in the 
art will be well familiar with the operation of the Internet, and the technical 
requirements for coupling a computer system therewith. 

The encoded physical medium 30 typically includes a plurality of regions 
such as region 32, 50, 52, and 54 wherein information has been encoded. The 
content of such regions may be encoded according to a well known content 
encoding scheme such as a bar code scheme. The present invention also teaches a 
variety of new encoding schemes. For example, a content encoding scheme 
contemplated by the present invention is a bar code printed using invisible, e,g. 
infrared (IR), inks. Such a bar code would be apparent to the sensor but invisible 
to the user. Alternatively, the content of the region 32 may be encoded according 
to a spectral encoding scheme. One specific example of a suitable spectral 
encoding scheme is described below with reference to Figures 6-8. The encoded 
physical medium 30 may also include a document identification (ID) hotspot 33, 
similar to the region 32. The content encoded within the document ID hotspot 33 
will, however, be special in that it provides an indication of the identity of the 
particular encoded physical medium 30. 

The encoded regions 50, 52, and 54 are special regions termed Linkmarks. 
Encoded within the Linkmark 50 is information that the computer system 40 can 
utilize to open, access or implement a computer implemented process such as a 
computer application, a web page, or a web site. In certain embodiments, 
information encoded on the Linkmark 50 may include a uniform resource locator 
(URL) that directs to a particular web page or web site located on the Internet. In 
other embodiments, information encoded on the Linkmark 50 may include a 
particular document available upon the computer system 40, the computer 
implemented process in this case being the application which must execute in 
order to access the particular document. Certain Linkmarks are Multicon 
Linkmarks which, in addition to encoded machine readable information, provide 
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human readable information indicating to a viewer the nature, function, or 
capability of the encoded machine readable information. Different embodiments 
of Linkmarks and their use are described in more detail below with reference to 
Figures 3-5 and Figures 9-12. 

5 In certain embodiments, encoded regions such as the region 32 also include 

a marker 34 indicating that certain encoded information is present in the region 32. 
By way of example, in one spectral encoding scheme, the desired content for the 
region 32 is represented according to different visible and infrared inks (reflective 
and/or photoluminescent) applied to the region 32. The marker 34 is represented 
10 by yet another infrared ink similarly applied to the region 34. In this example, the 
user would not see the marker 34, but may or may not see a visual representation 
of the content encoded in the region 32. 

The encoded physical medium 30 may take any suitable form. By way of 
example, the encoded physical medium 30 may be a page from a physical book or 

15 magazine, an article of clothing such as a T-shirt, a globe, consumer product 
packaging, etc. Such encoded physical mediums may have been marked and 
encoded with data for the specific purpose of providing the interface of the present 
invention. Altematively, the encoded physical medium 30 may simply be items 
selected from a user's surroundings, the encoded information arising randomly or 

20 unintentionally (at least with relation to the user's application). In another 
embodiment, the encoded content arises randomly or unintentionally, but a marker 
34 is applied intentionally. 

The sensor 12 includes a sensing element 13, a decoder 14, a transceiver 16, 
an on/off switch 18, and memory 20. The sensing element 13 is arranged to 

25 measure information present on the encoded physical medium 30. When the 
utilized encoding scheme implements a marker 34, the decoder 14 is arranged to 
determine whether the marker 34 is present in measured information, and when the 
marker 34 is sensed, to decode the measured information. The transceiver 16 is 
operable to transmit data such as decoded information to the computer system 40. 

30 Depending upon the embodiment, .the transceiver 16 may establish either a imi- 
directional or bi-directional communications link 22 between the interface 10 and 
the computer system 40. The communications link 22 is preferably a wireless 
communications link such as one based upon infrared (IR), radio-frequency (RF), 
or ultrasonic communications technology. However, the communications link 22 

35 may take the form of a wired communications link such as a serial RS-232 or RS- 
485 data cable, or a parallel data cable. 
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In some embodiments, the sensor 12 operates by measuring infomiation 
selected from the encoded physical medium 30 by the user, decoding as necessary, 
and then transmitting the decoded information to the computer system 40 for 
further processing. In other embodiments, the sensor 12 includes further hardware 
for processing the measured information locally. For example, the sensor 12 may 
include a microcontroller such as a PIC microcontroller programmed to process 
the measured information. The decoder 14 may be part of the microcontroller, or 
may be separate circuitry. In some embodiments, the sensor 12 maintains in the 
. memory 20 a database related to the measured information. The information 
stored in the database may be used locally at the sensor and/or saved for 
subsequent transmission. 

The computer system 40 appears in Figure 1 as a personal desktop 
computer. However, it is contemplated that the interface 10 is suitable for use 
with a wide scope of electronic devices. The wide scope of suitable computer 
systems encompasses all types of personal computers; interactive TV systems, set- 
top boxes, web interfaces, haptic interfaces, streaming music and video sources, 
and many others. One particular example is a WebTV net-top box." Further, 
although the interface 10 is intended to be representative of and thus generic to a 
broad range of interfaces contemplated by the present invention, it will be 
.appreciated that computer interfaces of the present invention may take many forms 
which go beyond the example interface 10 of Figure 1. 

With reference to Figure 2, a method 100 for providing an interface between 
a user and a computer system in accordance with one aspect of the present 
invention will now be described. An initial step 102 provides the user with an 
encoded physical medium 30 and a sensor 12 suitable for sensing information 
present within the encoded physical mediimi 30. As described above, the encoded 
physical medium 30 may take any suitable form. 

In a next step 104, the user explores the encoded physical medium 30 and 
selects a desired portion 32 of the encoded physical medium 30. The user may be 
drawn into the desired portion 32 through text, coloring, or graphics illustrated on 
the desired portion 32. The text, coloring or graphics illustrated on the desired 
portion 32 may represent the encoded information, may be in addition to the 
encoded information, or may be some suitable combination of illustration and 
encoded information. Instead of being drawn in, perhaps in the case of a game or 
work task, the user may be selecting the desired portion 32 based upon some 
predefined criteria. By way of example, the user may desire access to a particular 
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web page, wish to initiate a certain computer application, be searching for a clue to 
a puzzle game, or doing inventory and selecting a bar code found on a product 
during this process. In any event, once the user has selected the desired portion 32 
in step 104, in a step 106 the user engages the sensor 12 with the desired portion 
5 32 of the encoded physical medium 30, The sensor engagement of step 106 will 
typically involve the user setting the sensor 12 to an ON state that indicates that 
the sensor 12 should be in operation. In the case of the interface 10 of Figure 1, 
the step 106 would involve operation of the on/off switch 18. Depending upon the 
particular sensor and the application, sensor engagement may require the user to 
10 bring the sensor 12 into close proximity to or in contact with the desired region 32. 

In a next step 108, the sensor 12 measures information present within the 
desired region 32 of the encoded physical medium 30. Information is defined 
herein as any data that the sensor 12 is capable of measuring. Thus, the 
information measured by the sensor 1 2 is not limited to information that has been 
15 purposefully encoded into the desired region 32 of the encoded physical medium 
30. 

A step 110 then determines whether the measured information has null 
meaning. When step 110 determines that the measured information has null 
meaning, it is simply determining that the measured information has not been 

20 marked, for the present purposes, as containing encoded information. However, as 
will be appreciated, a determination of null meaning may be consequential. 
Accordingly, when step 110 deteraiines that the measured information has null 
meaning, a step 112 performs any action indicated by such a determination. For 
example, the sensor 12 may be equipped with a light that blinks or a buzzer that 

25 sounds when the measured information has null meaning. As another example, the 
sensor 12 may have memory 20 that is utilized to keep track of the meaning of the 
different measured regions, including each null determination. Alternatively, the 
null information may be transmitted to the computer system 40. In any event, 
once step 112 is complete, the control of the method 100 is passed back to step 

30 104 allowing the user to explore further and select another region 32 for sensing. 

When it is determined in step 1 10 that the information measured in step 108 
does not have null meaning (e.g., the measm-ed information has been marked as 
containing encoded information), control is passed to a step 114 wherein the 
sensor 12 interprets the measured information. Depending upon the specific 
35 application, step 1 14 may involve just decoding of the information from the 
particular encoding scheme into a data format suitable for transmission by the 
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transceiver 16. However, in some embodiments significant processing of the 
measured information will occur locally at the sensor 12. Thus in a next step 1 16, 
the sensor 12 performs an operation that is a function of both the information 
interpreted in the step 1 14 and the context in which the information was measured. 
Note that context depends upon the particular application and may include the 
nature of previously interpreted information, the timing of the user's engagement 
of the sensor 12, information received at the sensor from the computer system 40, 
etc. 

For example, v^th each new engagement of the sensor 12, the sensor 12 may 
store the interpreted information in a database in the memory 20 and then evaluate 
the database or a portion of it to determine whether a predefined condition has 
been satisfied. A predefined condition could be the user gathering a set number of 
clues or data points, at which point the sensor transmits all or some of the stored 
information to the computer system 40. In one specific example, the user may be 
perusing an encoded catalog magazine 30 with a hand-held wand sensor 12, As 
the user engages the wand sensor 12 with regions of the catalog 30 representing 
desired products, these regions are sensed and the information therein interpreted 
by the wand sensor 12. When finished, the user may select an order region 32 
indicating to the sensor that the user is ready to order and purchase the selected 
items. At this point, the communication link 22 coidd be established with the 
computer system 40, which may be local or remote, and the user's order 
information could be transmitted to the computer system 40 which in turn could 
process the order or fiirther transmit the order as necessary. 

In other embodiments, the indicated action of step 116 includes the 
computer system 40 and/or the sensor 12 responding to the measured information 
by providing feedback. The feedback could take any suitable form such as audio, 
visual or tactile feedback. In any event, once the indicated action has been 
performed in the step 1 16, the control of the method 100 is passed back to stq) 104 
allowing the user to further explore the encoded physical medium 30 and select 
other regions for sensing. 

As noted above with reference to Figure 1, certain encoded physical 
mediums 30 include a document ED hot spot 33. In these embodiments, when a 
user first begins exploring the encoded physical medium 30 as described in step 
104, the initial step 106 ought to be the engagement of the sensor 12 with the 
document ID hot spot 33. Then in steps 114-116, the sensor 12 could store the 
document ID for later use, or immediately transmit it to the computer system, or 
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both; it depends upon the specific application. For example, it is contemplated 
that the sensor 12 store the document ID and include it with content decoded from 
each subsequently measured region prior to further processing of the decoded 
content. 

5 Further, certain encoding schemes may not require the use of a marker. 

Within these schemes, steps 1 10 and 1 12 of Figure 2 would become unnecessary, 
and thus another method for providing an interface using an encoding scheme 
without a marker could be implemented by simply skipping directly from step 108 
to step 1 14 of Figure 2. 

10 With reference to Figures 3-5, a few different embodiments of the Linkmark 

50 will be described. In Figure 3, a Linkmark 52 includes a Linkmark 50, and two 
instruction marks 60 and 62, The Linkmark 50, as described above with reference 
to Figure 1, includes encoded information that serves to direct the computer 
system 40 to a web page, a web site, an application program, a certain computer 

15 document, or some other computer implemented process. Each of the instruction 
marks 60 and 62 include operating instructions that can be executed by or are 
related to control of the computer implemented process indicated by the Linkmark 
50, For instance, should selection of the Linkmark 50 result in the invocation of a 
particular web site, the instmction marks 60 and 62 may be navigation conmiands 

20 such as scroll up and scroll down commands, respectively. In another example, 
the Linkmark 50 could invoke an application executing on the computer system 40 
and the instruction marks correspond to particular fimctions (e.g., navigation such 
as scrolling, print docimient, select text, run macro, etc.) that could be performed 
by the application. 

25 Figures 4 shows a Linkmark 54 including a Linkmark 50 and four 

instruction marks 60-66. Figure 5 shows a Linkmark 56 including a Linkmark 50 
and six instruction marks 60-70. These each illustrate suitable embodiments that 
provide a user of the printable interface additional tools for controlling the 
computer process invoked by selecting the Linkmark 50. The possible variety of 

30 printable interface tools include commands such as back, forward, left, right, up, 
down, home, end, recent, favorite, options, open mail, scroll, page down, page up, 
zoom in, zoom out, invoke an alphanumeric input utility, alphanumeric characters, 
left and right panning, print screen, control, shift, enter, escape, help, more 
information, etc. 
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With reference to Figures 6-8, a spectral encoding scheme according to 
another embodiment of the present invention will be described. The spectral 
encoding scheme represents content via three different values encoded within a hot 
spot 220. The sensor " decodes" these values by measuring the intensities of three 
5 different encoding colors CI, C2, and C3 found within the hot spot 220. CI, C2, 
and C3 may, for example, correspond to red, green, and blue (RGB). 
Alternatively, CI, C2, and C3 may be selected from outside the visible light range 
(e.g., infrared colors) or may be a combination of visible and invisible colors. 

As will be apparent to those skilled in the art, the hot spot 220 itself can be 
10 created using inks whose colors do not correspond directly to CI, C2, and C3. 
Take the instance where CI, C2, and C3 correspond to RGB. Most likely, the 
color printing system selected to produce the hot spot 220 will be a "CMYK" type 
using cyan (C), magenta (M), yellow (Y), and black (K) inks to produce color 
images. In such a case, the encoded medium designer may be provided a mapping 
15 between CMYK space and the different content values, even though the sensor 
will be determining each content value by measuring the intensities of the three 
different encoding colors RGB within the hot spot 220. 

Figure 6 represents diagramattically a hot spot 220 encoded according to the 
spectral encoding scheme. A pie chart 222 indicates that the different encoding 

20 colors CI, C2, and C3 are measurable in the hot spot 220, each taking on their own 
particular intensity. Thus the engaged sensor would measure three different 
values, one each for CI, C2, and C3. These values taken together provide the 
encoded content. Figure 6 does not illustrate the visual appearance a hot spot 
would likely take on, but merely represents that the different encoding colors are 

25 measurable within the hot spot 220. Of course, depending upon the content 
encoded therein, each hot spot will have varying intensity levels and in some 
instances the intensity level of certain encoding colors would be zero. The actual 
visual appearance of the hot spot 220 would include any text and/or graphical 
illustrations that the designer has created. 

^ 30 Figure 7 illustrates a sensor 300 responsive to the spectral encoding scheme 

and thus operable to measure information from an encoded physical medium 30, 
The sensor 300 includes a light emitter 302, a sensing element 304, and a shroud 
306. The light emitter 302 includes three light emitting diodes LEDl, LED2, and 
LED3, each operable to emit light corresponding to CI, C2, and C3, respectively. 
35 The sensing element 304 is a broadband sensing element responsive to the entire 
light spectrum. A user engages the sensor 300 with a desired region 32 of the 
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encoded physical medium 30 by turning the sensor 300 on and bringing the light 
emitter 302 and the sensing element 304 into reasonably close proximity to the 
desired region 32, When the sensor 300 is properly engaged with the desired 
region 32, the shroud 306 helps prevent the sensing element 304 from measuring 
5 extraneous information in the form of ambient light. 

With reference to Figure 8, one suitable method 108 for measuring the 
information stored within the desired region 32 will now be described. Simply 
put, the method 108 of Figure 8 sequences through measuring the intensities of the 
encoding colors CI, C2, and C3. In a first step 320, the user engages the sensor 

10 300 with the desired region 32. A step 322 turns LEDl on, measures the reflected 
intensity of CI, and then turns LEDl off. A step 324 turns LED2 on, measures the 
reflected intensity of C2, and then turns LED2 off A step 326 turns LED3 on, 
measures the reflected intensity of C3, and then turns LED3 off. Typically the 
sensing element 304 will generate an analog voltage proportional to the light 

15 intensity and the sensor 300 will include an analog-to-digital (A/D) converter. 
Thus the number of content identification numbers available with the first 
encoding scheme is directly dependent upon the precision of the A/D converter. 

With reference to Figure 9, a method 400 for utilizing Linkmarks such as 
Linkmarks 50, 52, and 54 will now be described. In a step 402, the computer user 
20 selects a desired hot spot such as region 32. Then, in a step 404, the sensor 12 and 
the computer system 40 work together to sense, decode, and determine the nature 
of the certain information encoded within the selected hot spot. 

In a next step 406, the computer system determines whether the encoded 
information directs the computer system to a web page or web site. For example, 
25 the encoded information may include a xmiform resource locator (URL) indicating 
a specific web page. When the computer system 40 determines in step 406 that the 
encoded information does invoke a web page or web site, in a step 408 the 
computer system 40 establishes a communications link with the desired web page 
or web site, and provides the information to the user. For example, the web page 
1 30 would likely be displayed upon a display screen coupled to^ the computer system 
40. Once the commimications link is established and the web page displayed for 
the computer user, process control is retumed to step 402 and the computer system 
awaits another command from the computer user and/or continues performing any 
ongoing operations. 
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On the other hand, when the computer system 40 determines in step 406 that 
the encoded information does not invoke a web page or web site, then in a step 
410 the computer system 40 determines whether the encoded information indicates 
that an appHcation should be invoked or a document opened. If yes, in a step 412, 
5 the computer system 40 will invoke the application, or open the document with an 
appropriate application. After step 412, process control is returned to step 402 and 
the computer system awaits another command frorn the computer user and/or 
continues performing any ongoing operations. If the encoded information does 
not indicate that a computer implemented process should be invoked, process 
10 control is passed to a next step 414. 

In step 414, the computer system determines whether the encoded 
information provides an instruction command for a previously invoked computer 
process such as a web page or application. When the encoded information does 
provide an instmction command for a previously invoked computer process, in a 

15 step 416 the computer system determines whether to execute the instruction 
command locally or transmit the instruction command to the remote computer 
process for execution. When the instruction command is to be performed locally, 
step 416 performs such execution. In any event, after step 416, process control is 
returned to step 402 and the computer system awaits another command ft*om the 

20 computer user and/or continues performing any ongoing operations. 

When step 414 determines that the encoded information does not provide an 
instraction command for a previously invoked computer process, process control 
is passed to a step 418. In step 418, the computer system 40 determines the nature 
of the information provided by the encoded information and performs any 
25 necessary actions. Then, process control is returned to step 402 and the computer 
system awaits another command from the computer user and/or continues 
performing any ongoing operations. 

The foregoing has been mostly devoted towards a description of encoded 
machine readable inforaiation, including both the character of the encoded 

30 information and some suitable techniques for encoding the information in a 
machine readable format. For example, with reference to Figures 3-5, it was 
described above how the encoded machine readable information could in some 
instances provide a direction to a web page, and in related instances the encoded 
machine readable information provided instructions suitable for controlling 

35 operation of the web page. Similarly, with reference to Figures 6-8, one suitable 
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encoding scheme for machine readable infomiation, a spectral encoding scheme, 
was described. 

The present invention further contemplates that in addition to the encoded 
machine readable information, Multicon hot spots, links, or instruction marks will 
5 provide human readable information typically related to the encoded machine 
readable information. Take, for example, a simple color coding scheme 
distinguishing the nature of the instruction marks. In one suitable paradigm, 
different shades of green could indicate navigation instructions, different shades of 
red could indicate editing functions, etc. 

10 In certain embodiments, the Multicon Linkmarks provide graphical 

illustrations that indicate at least one characteristic of the computer process to 
which the machine readable encoded linking information directs the computer 
system. For example, the graphical illustration may indicate the media format 
available at a web page, the content type available fi:om the web page, the 

15 transmission rate at which the web page can commimicate, the type of user to 
which the web page may appeal, etc. 

With reference to Figure 10, a Multicon Linkmark 500 including human 
readable information will now be described. In the upper left-hand comer 502, 
human readable information regarding the media type available at the web site is 

20 shown. In the upper right-hand comer 504, human readable information regarding 
the content available at the web site is shown. In the lower right-hand comer 506, 
human readable information regarding the suggested user type and web page 
constraints is shown. In the lower left-hand comer 508, human readable 
information regarding the size of the web page or web site, as well as the 

25 communications rate is shown. 

The left-hand comer 502 has an audio icon 520, a text icon 522, and a video 
icon 524. The audio icon 520 has the appearance of a compact disk read only 
memory (CD-ROM) device (or perhaps a vinyl record). The text icon 522 has the 
appearance of a sheet of paper with text on it. The video icon 524 has the 
30 appearance of video film. These together indicate to the user that the web site has 
video, audio, and text available for access. The size of these icons in proportion to 
one another can be drawn to illustrate the proportion of different media type 
available at the web page. Additionally, other types of available media such as 
still photographs may be indicated by other icons 
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The upper right-hand comer 540 has a science icon 540, a sports icon 542, 
and a drama icon 544. The science icon 540 has the appearance of a partially 
filled beaker. The sports icon 542 has the appearance of a baseball. The drama 
icon 544 has the appearance of a couple of theater masks. These together indicate 
5 that sports, science, and drama content are all available from the web page. Of 
course, one can imagine a wide variety of different content that may be provided 
and a corresponding number of content icons. For example, a paintbrush and/or 
easel could represent art content, an image of capital building could indicate 
political content, an image of a bookshelf could indicate a library of information, 
10 an icon of a map could indicate geography information, etc. 

The lower right-hand comer 506 has a gender/age icon 560, a payment 
method icon 562, and a membership icon 564. The gender-age icon 560 has the 
appearance of an adolescent boy, thereby indicating the content available at the 
web page tends to be suited for male adolescents. The payment method icon 562 
15 has the appearance of a credit card, thereby indicating that in order to make 
purchases or take advantage of certain services available at the web page, the user 
must have a credit card account available for charging. The membership icon 564 
indicates that in order to fully utilize the web page, the user must be a member of a 
club or organization that controls the web page. 

20 The lower left-hand comer 508 has a size icon 580, a duration icon 582, and 

a communications speed icon 584. The size icon 580 has the appearance of a box, 
or storage cabinet, thereby indicating that an entire web site is available, as 
opposed to just a web page. Alternatively, the size icon 580 could have the 
appearance of a page of paper indicating that just a web page is available. The 

25 duration icon 582 has the appearance of a "shooting" star thereby indicating that 
content on the web page tends to be dynamic and temporary. Alternatively, the 
duration icon 582 could take the appearance of a sun, star, pyramid, or other such 
image that conveys permanence. The communications speed icon 584 has the 
appearance of three curves each with a single arrowhead. This might indicate a 

30 relatively slow communications rate. In contrast, a communications speed icon 
584 that had three curves each with multiple arrowheads may indicate a relatively 
rapid communications rate. 

As will be appreciated, the Multicon Linkmark can take on a variety of 
forms. In particular, the Multicon Linkmark is not limited to the four quadrant 
35 type Multicon Linkmark 500 of Figure 10, Rather, the human readable 
information can be arranged within the Linkmark in any order, including 

-18" 



wo 99/34277 



PCTAJS98/27210 



randomly. Further, a Multicon can be used separately from a Linkmark, For 
example, imagine an interface book full of different Linkmarks and Multicon 
Linkmarks. On the cover of the interface book (or perhaps on book tabs), it may 
be useful to display a reference Multicon that indicates the variety of different 
5 Linkmarks and Multicon Linkmarks available within the book, even though the 
reference Multicon need not itself include machine readable information. 

Figure 1 1 illustrates a Multicon Linkmark 500 having human readable 
information as described above with reference to. Figure 10, the Multicon 
Linkmark 500 coupled together with four different instmction marks 600, 602, 
10 604, and 606. The instruction marks include machine readable information 
providing operating instructions for controlling the computer process indicated by 
the information encoded in the Multicon Linkmark 500. Additionally, the 
instruction marks may be color or graphic design coded to provide an indication of 
the nature of the instruction in a human readable format. 

15 Figure 12 is a Multicon legend 618 showing some example Multicon 

Linkmark icons conveying human readable information. An icon 620 indicates 
computer related subject matter. An icon 622 indicates audio media. An icon 624 
indicates video media. An icon 626 indicates film media. Icons 628 and 630 
represent a quantity of information metric, small and large, respectively. An icon 

20 632 indicates that content is intended for adult and children audiences. An icon 
634 indicates that content is intended for adult audiences. An icon 636 indicates 
text media. An icon 638 represents entertainment related subject matter. Icons 
640 represent the "timeliness" or update frequency of information available at the 
corresponding web page. 

25 Thus under the scheme of Figure 12, the Multicon Linkmark 644 of Figure 

13 indicates to a user that the corresponding web site is entertaining, contains 
subject matter suitable for adults and children, includes film and audio media, and 
is a relatively small website that is updated irregularly. Likewise, the Multicon 
Linkmark 646 of Figure 14 indicates to the user that the corresponding web site is 

30 entertaining, contains subject matter suitable for adults and children, includes text 
and audio media, and is a relatively large website that is updated regularly. 
Finally, the Multicon Linkmark 648 of Figure 15 indicates to the user that the 
corresponding web site is computer-related, contains subject matter intended 
primarily for adults, consists of text media, and is a relatively small website that is 

35 updated regularly. 
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Figure 1 6 illustrates the rendering of two-dimensional bar codes using four 
different colors. A cyan glyph 650, a magenta glyph 652, a yellow glyph 654 and 
a black glyph 656 are combined together to form a colored two-dimensional bar 
code 660. Each of the single color glyphs is arranged such that the digital 
5 information represented by the combination two-dimensional bar code 660 
provides a meaningful two-dimensional bar code. Additionally, the intensities of 
the coloring in each single color glyph is designed such that the color average, as 
shown by the image 662, further stores information encoded as the color average. 
Thus information is stored within the two-dimensional bar code 660 both by the 
10 combined glyph and the color average. By providing a technique for dual 
encoding, the present invention means users may have sensors of varying 
sophistication. For example, certain sensors may only be capable of determining 
the average color information, while others may be able to determine both average 
color information and the information encoded in the two-dimensional bookmark. 

15 While this invention has been described in terms of several preferred 

embodiments and a number of specific examples, there are alterations, 
permutations, and equivalents which fall within the scope of this invention. 
Therefore it is desired that the appended claims be interpreted as including all such 
alterations, permutations, and equivalents as fall within the true spirit and scope of 

20 the present invention. 
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What is claimed is: 



CLAIMS 

1. An encoded physical medium suitable for use in interfacing a 
5 computer user and a computer system such that the user can control a plurality of 
computer implemented processes, the encoded physical medium comprising: 

a linkmark wherein machine readable linking information has been encoded, 
the encoded linking information indicating a computer implemented process, the 
computer system operable to open up the computer implemented process when the 
10 linkmark is selected by the computer user; and 

an instmction mark wherein a machine readable operating instruction has 
been encoded, the encoded operating instmction including an instruction that can 
be executed by the computer implemented process. 



15 2. An encoded physical medium as recited in claim 1 wherein the 

computer implemented process corresponds to a web page. 



3- An encoded physical medium as recited in claim 2 wherein the 
encoded operating instruction is a navigation instmction suitable for controlling 
20 navigation on the web page. 



4. An encoded physical medium as recited in claim 3 wherein the 
encoded operating instruction is a scroll command. 



25 5. An encoded physical medium as recited in claim 2 wherein the 

linkmark further includes human readable information providing an indication of 
at least one characteristic of the web page to which the encoded linking 
information directs the computer system. 
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6. An encoded physical medium as recited in claim 5 wherein the human 
readable information provides a graphical indication of media format available at 
the web page. 

5 

7. An encoded physical medium as recited in claim 6 wherein the media 
format available at the web page includes playable video. 



8. An encoded physical medium as recited in claim 7 wherein the 
10 graphical indication of the media format available at the web page includes a 
graphic of videotape recording media. 



9. An encoded physical mediimi as recited in claim 6 wherein the media 
format available at the web page includes playable audio. 

15 

10. An encoded physical medium as recited in claim 9 wherein the 
graphical indication of the media format available at the web page includes a 
graphic of an audio recording media. 



20 11, An encoded physical medium as recited in claim 6 wherein the media 

format available at the web page includes text. 



12. An encoded physical medium as recited in claim 11 wherein the 
graphical indication of the media format available at the web page includes a 

25 graphic corresponding to text. 

13. An encoded physical medium as recited in claim 6 wherein media 
format available at the web page includes at least two of video, audio, and text. 
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14. An encoded physical medium as recited in claim 13 wherein the 
graphical indication includes a separate graphic for each type of media format 
available at the web page, and the relative physical sizing of each separate graphic 
5 one to another provides an indication of the relative proportion of each type of 
media format available at the web page. 



15. An encoded physical medium as recited in claim 5 wherein the human 
readable information provides a graphical indication of content type available 

1 0 from the web page. 

16. An encoded physical medium as recited in claim 15 wherein the 
content type available from the web page includes at least one content type 
selected from the group consisting of arts, humanities, business, economy, 

15 computers, Intemet, education, entertaiimient, government, health, news, media, 
recreation, sports, reference, regional, science, social science, society, culture, and 
geography. 

1 7. An encoded physical medium as recited in claim 5 wherein the human 
20 readable information provides a graphical indication of a conmiunications rate that 

the computer user can expect to experience when interfacing with the web page. 

18. An encoded physical medium as recited in claim 5 wherein the hxmian 
readable information provides a graphical indication of the quantity of information 

25 available at the web page. 

19. An encoded physical medium as recited in claim 5 wherein the human 
readable information provides a graphical indication of at least one requirement 
that must be met in order to fully utilize the web page. 

30 
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20. An encoded physical medium as recited in claim 19 wherein the at 
least one requirement that must be met in order to fully utilize the web page is that 
the computer user must be a member. 



21. An encoded physical medium as recited in claim 19 wherein the at 
least one requirement that must be met in order to fully utilize the web site is that 
the computer user must have a credit card available. 



22. An encoded physical medium as recited in claim 19 wherein the at 
least one requirement that must be met in order to fully utilize the web site is that 
the computer system must be capable of miming a specific computer application. 



23. An encoded physical medium as recited in claim 5 wherein the human 
readable information provides a graphical indication of a user type that may be 
particularly interested in the web page. 



24. An encoded physical medium as recited in claim 23 wherein the 
graphical indication of the user type indicates a suggested user age range. 



25. An encoded physical medium as recited in claim 23 wherein the 
graphical indication of the user type indicates a suggested user gender. 



26. An encoded physical medium as recited in claim 1 wherein the 
computer implemented process corresponds to an application program that is 
executable upon the computer system. 



27. An encoded physical medium as recited in claim 26 wherein the 
encoded operating instruction is a navigation instruction suitable for controlling 
navigation within the application program. 
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28. An encoded physical medium as recited in claim 27 wherein the 
encoded operating instruction is a scroll command. 



5 29. An encoded physical medium as recited in claim 26 wherein the 

encoded operating instruction is a particular function that can be performed by the 
application program. 



30. An encoded physical medium as recited in claim 1 wherein the 
10 encoded information has been encoded according to a spectral encoding scheme. 



31, An encoded physical medium as recited in claim 30 wherein the 
spectral encoding scheme includes light outside the visible light spectrum. 

15 32. An encoded physical medium as recited in claim 31 wherein the 

spectral encoding scheme includes infrared light. 

33. An encoded physical medium as recited in claim 30 wherein the 
spectral encoding scheme includes encoding colors red, green, and blue, and the 
20 spectral encoding scheme encodes information based upon the intensities of the 
encoding colors red, green, and blue. 



34. An encoded physical medium as recited in claim 1 wherein the 
encoded information has been encoded according to a two-dimensional bar code. 

25 

35. An encoded physical medium as recited in claim 1 wherein the two- 
dimensional bar code is generated using a spectral encoding scheme such that the 
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average color of each mark corresponds to a certain information separate from the 
bar code information of each mark. 



36. A computer system suitable for controlling a computer implemented 
process, the computer system comprising: 

a digital processor; 

memory coupled to the digital processor, the memory storing operating 
instructions suitable for controlling the operation of the digital processor; 

an encoded physical medium including: 

a linkmark wherein machine readable linking information has 
been encoded, the encoded linking information indicating a computer 
implemented process, the computer system operable to open up the 
computer implemented process when the linkmark is selected by the 
computer user; and 

an instmction mark wherein a machine readable operating 
instruction has been encoded, the encoded operating instruction being 
an instmction that can be executed by the computer implemented 
process; and 

a sensor coupled to the digital processor and operable for measuring 
information present within each mark located on the encoded physical medium. 



37. A computer system as recited in claim 36 wherein the coupling 
between the sensor and the digital processor is a remote linkage. 



38. A computer system as recited in claim 36 wherein the computer 
system is coupled to the Internet, and the encoded linking information includes a 
uniform resource locator (URL). 
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39. A computer system as recited in claim is a WebTV coupled to the 
Internet, and the encoded linking information includes a uniform resource locator 
(URL). 



5 40. A two-dimensional bar code comprising a plurality of colored glyphs, 

each colored glyphs having a color intensity such that information is encoded as an 
average color intensity of the combination of the plurality of colored glyphs, the 
plurality of colored glyphs also arranged such that digital information is encoded 
within the two-dimensional bar code. 

10 

41. A two-dimensional bar code as recited in claim 40 wherein the 
plurality of colored glyphs includes a cyan glyph. 

42. A two-dimensional bar code as recited in claim 40 wherein the 
15 plurality of colored glyphs includes a magenta glyph. 

43. A two-dimensional bar code as recited in claim 40 wherein the 
plurality of colored glyphs includes a yellow glyph. 

20 44. A two-dimensional bar code as recited in claim 40 wherein the 

plurality of colored glyphs includes a black glyph. 

45. A two-dimensional bar code as recited in claim 44 wherein the 
plurality of colored glyphs further includes cyan, magenta, and yellow glyphs. 

25 

46. A method for interfacing a human user with a computer system, the 
method comprising the steps of: 
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providing an encoded physical medium including a linkmark wherein 
machine readable linking information has been encoded, the encoded linking 
information indicating a computer implemented process; 

in response to the user selecting the linkmark, measuring and decoding the 
machine readable information encoded within the link; and 

invoking the computer implemented process. 



47. A method as recited in claim 46 wherein the machine readable linking 
information includes a uniform resource locator, and the computer implemented 
process indicated by the machine readable linking information is a web page. 



48, A method as recited in claim 47 wherein the step of invoking the 
computer implemented process includes the substeps of: 

establishing a communications link between the computer system and the 
web page; and 

displaying the web page upon a display screen of the computer system. 



49. A method as recited in claim 48 wherein the encoded physical 
medium further includes an instruction mark wherein a machine readable 
operating instruction has been encoded, the encoded operating instruction 
including an instmction that can be executed by the computer implemented 
process. 



50. A method as recited in claim 49 where, in response to the user 
selecting the instruction mark, the computer system decodes the machine readable 
operating instruction and transmits it to the web page. 



51. A method as recited in claim 50 wherein the web page performs the 
machine readable operating instruction. 
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52. A method as recited in claim 51 wherein the machine readable 
operating instruction is navigation instruction. 



5 53. A method as recited in claim 46 wherein the computer implemented 

process is a computer application, and the step of invoking the computer 
implemented process includes initiating execution of the computer application on 
the computer system. 



10 54. A method as recited in claim 53 wherein the encoded physical 

medium further includes an instmction mark wherein a machine readable 
operating instruction has been encoded, the encoded operating instruction 
including an instruction that can be executed by the computer application. 



15 55. A method as recited in claim 54 where, in response to the user 

selecting the instruction mark, the computer application performs the encoded 
operating instruction. 



56. An encoded physical medium suitable for use in interfacing a 
20 computer user and a computer system such that the user can invoke and interface 
with a web site by way of the encoded physical medium and the computer system, 
the encoded physical medium comprising a linkmark wherein machine readable 
linking information has been encoded, the encoded linking information indicating 
a particular web site, the computer system operable to establish a communications 
25 link with the web site when the linkmark is selected by the computer user, the 
linkmark displaying human readable information in the form of at least one 
graphical icon, the at least one graphical icon providing an indication of at least 
one characteristic of the particular web page to which the encoded linking 
information directs the computer system. 

30 
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57. An encoded physical medium as recited in claim 56 wherein the 
graphical icon is an image of a video camera recorder thus indicating that the 
particular web page has content available in video film format. 



58. An encoded physical medium as recited in claim 56 wherein the 
graphical icon is a computer system image thus indicating that the particular web 
page has content related to computer technology. 



59. An encoded physical medium as recited in claim 56 wherein the 
graphical icon is an image of an audio speaker thus indicating that the particular 
web page has content available in audio media format. 



60. An encoded physical medium as recited in claim 56 wherein the 
graphical icon is an image of a child thus indicating that the particular web page 
has content that is suitable for children. 



61. An encoded physical medium as recited in claim 56 wherein the 
graphical icon is an image of an adult thus indicating that the particular web page 
has content that is suitable for adults. 
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PROVIDE TERMINAL 
PAGE INFORMATION TO 
USER 
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DOES THE ENCODED 
INFORMATION PROVIDE A 
NAVIGATION OR FUNCTION 
COMMAND? 



I 

Yes 



416 



EXECUTE NAVIGATION OR 
FUNCTION COMMAND ON 

CURRENT ACTIVE 
DOCUMENT, APPLICATION 
OR TERMINAL PAGE 



Figure 9 
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Figure 16 




